Information notification apparatus that notifies information of motion of a subject

ABSTRACT

An information notification apparatus ( 1 ) includes: an image acquisition unit ( 42 ) that acquires a plurality of moving images in which motions of subjects are captured, respectively; a first detection unit ( 44 ) that detects a specific state in the motions of the subject from respective moving images among the plurality of moving images acquired by the acquisition unit; a face detection unit ( 45 ) that detects an image corresponding to the specific state detected by the first detection unit from the respective moving images; a comparison unit ( 47 ) that compares images detected by the image detection unit; and a notification control unit ( 50 ) that notifies information based on a comparison result by the comparison unit.

This application is based on and claims the benefit of priority fromJapanese Patent Application No. 2012-280126, filed on 21 Dec. 2012, thecontent of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an information notification apparatusthat notifies information of a motion of a subject, an informationnotification method, and a storage medium.

Related Art

Japanese Unexamined Patent Application, Publication No. 2004-321767discloses a technology whereby an external apparatus is provided so thatthe apparatus faces a user across a golf ball for checking a user'smotion of a golf swing as a conventional technology.

SUMMARY OF THE INVENTION

An apparatus according to a first aspect of the present application isan information notification apparatus including: an acquisition unitthat acquires a plurality of moving images in which motions of subjectsare captured, respectively; a state detection unit that detects specificstates of the motions of the subjects from respective moving imagesamong the plurality of acquired moving images; an image detection unitthat detects images corresponding to the respective detected specificstates from the moving images; a comparison unit that compares thedetected images; and a notification control unit that notifiesinformation based on a comparison result by the comparison unit. Amethod according to a second aspect of the present application is amethod executed by an information notification apparatus, the methodincluding: an acquisition step of acquiring a plurality of moving imagesin which motions of subjects are captured, respectively; a statedetection step of detecting specific states of the motions of thesubjects from respective moving images among the plurality of acquiredmoving images; an image detection step of detecting images correspondingto the respective detected specific states from the moving images; acomparison step of comparing the detected images; and a notificationcontrol step of notifying information based on a comparison result bythe comparison unit. A non-transitory storage medium according to athird aspect of the present application is a non-transitory storagemedium encoded with a computer-readable program that enables a computerto execute functions as: an acquisition unit that acquires a pluralityof moving images in which motions of subjects are captured,respectively; a state detection unit that detects specific states of themotions of the subjects from respective moving images among theplurality of acquired moving images; an image detection unit thatdetects images corresponding to the respective detected specific statesfrom the moving images; a comparison unit that compares between thedetected images; and a notification control unit that notifiesinformation based on a comparison result by the comparison unit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a hardware configuration of aninformation notification apparatus 1 according to an embodiment of thepresent invention;

FIG. 2 is a functional block diagram showing a functional configurationfor executing comparison result notification processing, among thefunctional configurations of the information notification apparatus 1 inFIG. 1;

FIG. 3 is a diagram showing a coordinate system of a captured image;

FIG. 4 is a diagram showing a positions and the like of face images forthe player and the professional golfer for each of the seven specificswing state images;

FIG. 5 is a diagram showing an example of a notification image;

FIG. 6 is a diagram showing an example of a notification image;

FIG. 7 is a diagram showing an example of a notification image;

FIG. 8 is a diagram showing an example of a notification image;

FIG. 9 is a diagram showing an example of a notification image;

FIG. 10 is a flowchart showing an example of a flow of comparison resultnotification processing executed by the information notificationapparatus 1 of FIG. 1 having the functional configuration of FIG. 2; and

FIG. 11 is a diagram showing an example of a notification image.

DETAILED DESCRIPTION OF THE INVENTION

In the following, embodiments of the present invention are explainedwith reference to the drawings.

Configuration

FIG. 1 is a block diagram showing a hardware configuration of aninformation notification apparatus according to an embodiment of thepresent invention.

The information notification apparatus 1 is configured as, for example,a digital camera.

The information notification apparatus 1 includes a CPU (CentralProcessing Unit) 11, ROM (Read Only Memory) 12, RAM (Random AccessMemory) 13, a bus 14, an Input/Output interface 15, an image captureunit 16, an input unit 17, an output unit 18, a storage unit 19, acommunication unit 20, and a drive 21.

The CPU 11 executes various processing according to programs that arerecorded in the ROM 12, or programs that are loaded from the storageunit 19 to the RAM 13.

The RAM 13 also stores data and the like necessary for the CPU 11 toexecute the various processing, as appropriate.

The CPU 11, the ROM 12 and the RAM 13 are connected to one another viathe bus 14. The input/output interface 15 is also connected to the bus14. The image capture unit 16, the input unit 17, the output unit 18,the storage unit 19, the communication unit 20, and the drive 21 areconnected to the input/output interface 15.

The image capture unit 16 includes an optical lens unit and an imagesensor (not illustrated).

In order to photograph a subject, the optical lens unit is configured bya lens such as a focus lens and a zoom lens for condensing light.

The focus lens is a lens for forming an image of a subject on the lightreceiving surface of the image sensor. The zoom lens is a lens thatcauses the focal length to freely change in a certain range.

The optical lens unit also includes peripheral circuits to adjustsetting parameters such as focus, exposure, white balance, and the like,as necessary.

The image sensor is configured by an optoelectronic conversion device,an AFE (Analog Front End), and the like.

The optoelectronic conversion device is configured by a CMOS(Complementary Metal Oxide Semiconductor) type of optoelectronicconversion device and the like, for example. Light incident through theoptical lens unit forms an image of a subject in the optoelectronicconversion device. The optoelectronic conversion deviceoptoelectronically converts (i.e. captures) the image of the subject,accumulates the resultant image signal for a predetermined timeinterval, and sequentially supplies the image signal as an analog signalto the AFE.

The AFE executes a variety of signal processing such as A/D(Analog/Digital) conversion processing of the analog signal. The varietyof signal processing generates a digital signal that is output as anoutput signal from the image capture unit 16.

Such an output signal of the image capture unit 16 is hereinafterreferred to as “data of a captured image”. Data of a captured image issupplied to the CPU 11, RAM 13, etc. as appropriate.

Here, in the present embodiment, data of a single captured image issequentially output every 1/30 seconds, for example, to constitute dataof a moving image. In other words, in the following, a captured imagerefers to, unless otherwise noted, a unit image (a frame image, a fieldimage, etc.) constituting a moving image.

The input unit 17 is configured by various buttons and inputs a varietyof information in accordance with instruction operations by the user.

The output unit 18 is configured by the display unit, a speaker, and thelike, and outputs images and sound.

The storage unit 19 is configured by DRAM (Dynamic Random Access Memory)or the like, and stores data of various images.

The communication unit 20 controls communication with other devices (notshown) via networks including the Internet.

A removable medium 31 composed of a magnetic disk, an optical disk, amagneto-optical disk, semiconductor memory or the like is installed inthe drive 21, as appropriate. Programs that are read via the drive 21from the removable medium 31 are installed in the storage unit 19, asnecessary. Similarly to the storage unit 19, the removable medium 31 canalso store a variety of data such as the image data stored in thestorage unit 19.

FIG. 2 is a functional block diagram showing a functional configurationfor executing notification processing, among the functionalconfigurations of such an information notification apparatus 1.

The notification processing refers to a sequence of processing ofcomparing an analysis result of motions of a subject included in movingimages captured with an analysis result of motions such as of a personas a target for comparison, and generating and notifying informationbased on the result thus compared.

In the present embodiment, an example is explained of comparinginformation based on an analysis result of a golf swing of a player(hereinafter, referred to as “an analysis result from a player”)included in a moving image captured by the information notificationapparatus 1 with information based on an analysis result of a golf swingof a professional golfer (hereinafter, referred to as “an analysisresult of a professional golfer”) included in moving image data capturedby the information notification apparatus 1. However, a variety ofinformation can also be compared such as comparing information based onan analysis result of a recent golf swing of a player included in amoving image captured by the data comparison apparatus 1 withinformation based on an analysis result of a golf swing of the playerincluded in a moving image captured in the past.

When the notification processing is executed, as shown in FIG. 2, animage capture control unit 41, an image acquisition unit 42, an imagereading unit 43, a first detection unit 44, a face detection unit 45, asecond detection unit 46, a comparison unit 47, an advice informationgeneration unit 48, a notification image generation unit 49, and anotification control unit 50 function in the CPU 11.

However, FIG. 2 is merely an example, and at least a part of thefunctions of the CPU 11 may be imparted to hardware that performs imageprocessing such as a GA (Graphics Accelerator).

When the notification processing is executed, a first image storage unit61 and a second data storage unit 62 which are provided as areas of thestorage unit 19 are used.

In the image storage unit 61, a moving image (a plurality of capturedimages) that is output from the first image capture unit 16 and acquiredby the image acquisition unit 42 is stored.

In the second image storage unit 62, a moving image of a golf swing of aprofessional golfer is stored in advance.

The image capture control unit 41 sets various image capture conditionsinputted via the input unit 17 and controls the operation of imagecapture at the image capture unit 16.

In the present embodiment, in order to capture a subject (player) whomakes a sequence of motions, the image capture control unit 41 causesthe image capture unit 16 to capture the subject continuously andoutputs as data of a moving image.

The image acquisition unit 42 acquires data of a moving image outputfrom the image capture unit 16. Then, the image acquisition unit 42encodes data of a moving image and stores in the first image storageunit 61.

The image reading unit 43 reads data of a moving image of a playerstored in the first image storage unit 61 and data of a moving image ofa professional golfer stored in the second image storage unit 62. Inother words, the image reading unit 43 sequentially decodes data of amoving image stored in the first image storage unit 61 and the secondimage storage unit 62 with each captured image as a unit image andstores data of a sequence of captured images thus decoded in the RAM 13.

The first detection unit 44 detects, for each of a player andprofessional golfer, an image of a specific swing state based on thedata of the moving image of the player and the data of the moving imageof the professional golfer read by the image reading unit 43. Here, thespecific swing state refers to the seven states of “address” state,“halfway back” state, “halfway down” state, “top” state, “impact” state,“follow” state, and “finish” state, as shown in FIG. 4 described later.

More specifically, the first detection unit 44 extracts image data of ashaft part of a golf club based on the player's moving image data andthe professional golfer's moving image data. Furthermore, the firstdetection unit 44 executes Hough transform based on the image dataextracted and detects an angle made by a horizontal plane of the shaftpart of the golf club based on a result of the execution. Moreover, thefirst detection unit 44 detects, for each of the player and theprofessional golfer, images of the abovementioned seven specific swingstates based on the angle detected.

Furthermore, the first detection unit 44 stores, for each of the playerand professional golfer, the images of the seven specific swing statesdetected in the RAM 13.

The face detection unit 45 detects an image of a face portion of each ofthe player and professional golfer (hereinafter, referred to as “faceimage”) for each of the seven specific swing states detected by thefirst detection unit 44. With a face detection unit (not illustrated)being provided, the detection of the face image is executed based on anoutput from the face detection unit.

The second detection unit 46 detects a position coordinate of the faceimage, which is detected by the face detection unit 45, in a capturedimage for each of the seven specific swing state images. Morespecifically, the second detection unit 46 detects a position coordinateof the face image in a coordinate system of a captured image shown inFIG. 3. FIG. 3 is a diagram showing a coordinate system of a capturedimage. A coordinate of a captured image can be defined as a coordinateon an X-Y plane. However, in FIG. 3, the Z-axis is shown for the purposeof convenience in order to define a tilt of a face in a right-leftdirection described later.

FIG. 4 is a diagram showing positions and the like of face images forthe player and the professional golfer for each of the seven specificswing state images. The position coordinate of the face image detectedis represented with the parameter “c” of FIG. 4. For example, in the“address” state of the professional golfer in FIG. 4, it is shown thatthe position coordinate is X=33 and Y=18.

It should be noted that the second detection unit 46 associates theposition coordinate of the face image detected with each of the sevenspecific swing state images and stores the data in the RAM 13.

Furthermore, the second detection unit 46 detects the tilt of a face inthe right-left direction, the tilt of the face in the up-down direction,and the tilt of the face in the rotational direction, for each of theplayer and professional golfer, based on the face images thus detected.Here, the vector in the front direction of the face and the Z-axis isparallel, the tilts of the right-left direction and the up-downdirection of the face are zero degrees. With this case as a reference,the tilt in the right-left direction of the face is represented by arotational angle around the Y-axis of a projection vector onto the Z-Xplane of the vector in the front direction of the face, and the tilt inthe up-down direction of the face is represented by a rotational anglearound the X-axis of a projection vector onto the Y-Z plane of thevector in the front direction of the face.

Furthermore, when the vector in the front direction of the face isparallel with the Y-axis, the tilt of the face is zero degrees. Withthis case as a reference, the tilt of the face is represented by arotational angle around the Z-axis of a projection vector onto the X-Yplane of the vector in the front direction of the face.

The right-left orientation of the face and the up-down tilt of the facethus detected are represented with the parameter “a” in FIG. 4. Forexample, in the “halfway back” state of the professional golfer in FIG.4, it is shown that the face tilts 5 degrees in the right direction and33 degrees in the lower direction.

Furthermore, the tilt of the face in the rotational direction thusdetected is represented with the parameter “b” in FIG. 4. For example,in the “top” state of the professional golfer in FIG. 4, it is shownthat the tilt of the face is 11 degrees.

It should be noted that the second detection unit 46 associates the tiltof the face in the right-left direction detected, the tilt of the facein the up-down direction detected, and the tilt of the face in therotational direction detected with each of the seven specific swingstate images, and stores the data in the RAM 13.

Furthermore, the second detection unit 46 detects the size of a faceimage and a position coordinate of an image of a golf ball (hereinafter,referred to as “golf ball image”) in a captured image. The size of aface image is detected as an area of a face image in a captured image.With a golf ball detection unit (not illustrated) being provided, thedetection of a position of a golf ball image in a captured image isexecuted based on an output from the golf ball detection unit.

The size of a face image thus detected is represented with the parameter“d” in FIG. 4. For example, in a case of the professional golfer of FIG.4, it is shown that the size of the face image is 9.5.

Furthermore, the position coordinate of the golf ball image thusdetected is represented with the parameter “e” in FIG. 4. For example,in a case of the professional golfer of FIG. 4, it is shown that theposition of the golf ball image is X=38 and Y=78.

It should be noted that the second detection unit 46 stores the size ofthe face image and the position coordinate of the golf ball image in thecaptured image for each of the player and the professional golfer in theRAM 13.

The comparison unit 47 compares, between the player and the professionalgolfer, each value of the position coordinate of the face image, thetilts of the face in the right-left direction, the tilts of the face inthe up-down direction, and the tilts of the face in the rotationaldirection, detected by the second detection unit 46, and calculates thedifferences.

The advice information generation unit 48 generates advice informationbased on each of the difference values, which is a comparison result bythe comparison unit 47. The advice information refers to characterinformation notified to the player. The contents of the adviceinformation include information based on the difference values of theposition of the face image between the player and the professionalgolfer, i.e. a message such as “Move your head 15 cm lower and 15 cm tothe right” shown in FIG. 5. Furthermore, the advice informationgeneration unit 48 associates the advice information generated with theplayer's swing state as “position advice information” and stores thedata in the RAM 13.

Furthermore, the contents of the advice information in another aspectinclude information based on each difference value of the positioncoordinates of the face image, the tilts of the face in the right-leftdirection, the tilts of the face in the up-down direction, the tilts ofthe face in the rotational direction, between the player and theprofessional player, i.e. a message such as “Angle Difference:right/left: 25 degrees, up/down: 27 degrees, tilt: 19 degree,professional golfer position: 15 cm lower and 15 cm to the right” shownin FIG. 6. Furthermore, the advice information generation unit 48associates the advice information thus generated with the player's swingstate as “advice information relating to position and angle difference”and stores the data in the RAM 13.

Furthermore, the contents of the advice information in another aspectare “14 cm” which is a difference value between the X coordinate of thegolf ball image 85 and the X coordinate of the player's face image, “31cm” which is a difference value between the X coordinate of the golfball image 86 and the X coordinate of the professional golfer's faceimage; and a “difference: 17 cm” that is information based on thedifference between these difference values, as shown in FIG. 7, forexample. Moreover, the advice information generation unit 48 associatesthe advice information thus generated with the player's swing state as“advice information relating to difference between balls” and stores thedata in the RAM 13.

Furthermore, the contents of the advice information in another aspectinclude information based on transitions of the position coordinates ofthe player's face image for each swing state, i.e. the message such as“Your head moves too high from address to top” shown in FIG. 9, forexample. Furthermore, the advice information generation unit 48 storesthe advice information thus generated in the RAM 13 as “graph adviceinformation”.

The notification image generation unit 49 generates the data of anotification image based on advice information generated by the adviceinformation generation unit 48, the seven specific swing state imagesfor each of the player and the professional golfer detected by the firstdetection unit 44, and the position coordinate of the face image, thetilt of the face in the right-left direction, the tilt of the face inthe up-down direction, the tilt of the face in the rotational direction,and the position coordinate of the golf ball image, detected by thesecond detection unit 46.

For example, when the data of the notification image shown in FIG. 5 isgenerated, the notification image generation unit 49 extracts, from theRAM 13, data of an image in the “top” state of the player detected bythe first detection unit 44 and extracts, from the RAM 13, “positionadvice information” of the player's “top” state. Furthermore, thenotification image generation unit 49 generates each piece of data of amark image 81 representing the position of the face image and the frontdirection of the face of the player and a mark image 82 representing theposition of the face image and the front direction of the face of theprofessional golfer, based on the position coordinate of the face image,the tilt of the face in the right-left direction, and the tilt of theface in the up-down direction, detected by the second detection unit 46.Then, the notification image generation unit 49 generates, as the dataof the notification image shown in FIG. 5, image data including an imageof the player's “top” state in which the mark image 81 is superimposedon the player's face image, the “position advice information” stored tobe associated with the player's “top” state extracted from the RAM 13,and the mark images 81 and 82.

Furthermore, for example, when the data of the notification image shownin FIG. 6 is generated, the notification image generation unit 49extracts, from the RAM 13, the data of the “top” state images of theplayer and the professional golfer detected by the first detection unit44 and extracts, from the RAM 13, “advice information relating toposition and angle difference” of the player's “top” state. Furthermore,the notification image generation unit 49 generates each piece of dataof a mark image 83 representing the position of the face image and thefront direction of the face of the player and a mark image 84representing the position of the face image and the front direction ofthe face of the professional golfer, based on the position coordinate ofthe face image, the tilt of the right-left direction of the face, andthe tilt of the up-down direction of the face, detected by the seconddetection unit 46. Then, the notification image generation unit 49generates, as data of a left half region of the notification image shownin FIG. 6, image data including the mark image 83 and the player's “top”state image in which the mark image 83 is superimposed on the face imageof the player. Furthermore, the notification image generation unit 49generates, as data of the right half region of the notification imageshown in FIG. 6, image data including the mark image 84 and theprofessional golfer's “top” state image in which the mark image 84 issuperimposed on the face image of the professional golfer. Moreover, thenotification image generation unit 49 combines the “advice informationrelating to position and angle difference” stored to be associated withthe player's “top” state extracted from the RAM 13, the data of the lefthalf region of the notification image, and the data of the right halfregion of the notification image to generate the notification image datashown in FIG. 6.

Furthermore, for example, when the notification image data shown in FIG.7 is generated, the notification image generation unit 49 extracts, fromthe RAM 13, “halfway down” state images of the player and theprofessional golfer detected by the first detection unit 44 andextracts, from the RAM 13, “advice information relating to differencebetween balls” in the player's “halfway down” state. Furthermore, thenotification image generation unit 49 generates, as data of a left halfregion of the notification image shown in FIG. 7, image data including adifference value between the X coordinate of a golf ball image includedin the “advice information relating to difference between balls” storedby being associated with the player's “halfway down” state extractedfrom the RAM 13 and the X coordinate of the player's face image(hereinafter, referred to as “first coordinate difference value”) and animage of the player's “halfway down” state. Furthermore, thenotification image generation unit 49 generates, as data of a right halfregion of the notification image shown in FIG. 7, the difference valuebetween the X coordinate of the golf ball image included in the “adviceinformation relating to difference between balls” and X coordinate ofthe face image of the professional golfer (hereinafter, referred to as“second coordinate difference value”) and an image of the professional“halfway down” state. Furthermore, the notification image generationunit 49 combines a difference value between the first coordinatedifference value and the second coordinate difference value, the data ofthe left half region of the notification image, and the data of theright half region of the notification image to generate the notificationimage data shown in FIG. 7.

Furthermore, for example, when the notification image data shown in FIG.8 is generated, the notification image generation unit 49 extracts, fromthe RAM 13, image data of the “top” and “address” states for each of theplayer and the professional golfer, detected by the first detection unit44. Furthermore, the notification image generation unit 49 generates amark image 87 representing the position of the face image and the frontdirection of the face in the player's “address” state, a mark image 88representing the position of the face image and the front direction ofthe face in the player's “top” state, a mark image 89 representing theposition of the face image and the front direction of the face in theprofessional golfer's “top” state, and a mark image 90 representing theposition of the face image and the front direction of the face in theprofessional golfer's “top” state, based on the position coordinate ofthe face image, the tilt of the face in the right-left direction, andthe tilt of the face in the up-down direction, detected by the seconddetection unit 46.

Then, the notification image generation unit 49 superimposes the markimage 87 onto the face image in the player's “address” state andsuperimposes the mark image 88 onto the face image in the player's “top”state to generate data of an image including the mark images 87 and 88and the player's “address” and “top” states as data of the left halfregion of the notification image shown in FIG. 8. In addition, thenotification image generation unit 49 superimposes the mark image 89onto the face image in the professional golfer's “address” state andsuperimposes the mark image 90 onto the professional golfer's “top”state to generate data of an image including the marks 89 and 90 and theprofessional golfer's “address” and “top” states as data of the righthalf region of the notification image shown in FIG. 8. Furthermore, thenotification image generation unit 49 combines the data of the left halfregion of the notification image with the data of the right half regionof the notification image to generate the notification image data shownin FIG. 8.

Furthermore, for example, when the notification image data shown in FIG.9 is generated, the notification image generation unit 49 reads positioncoordinates of the player's and the professional golfer's face images inthe seven specific swing states which are detected by the seconddetection unit 46 and stored in the RAM 13. Furthermore, thenotification image generation unit 49 extracts the player's “adviceinformation relating to graph” from the RAM 13. Then, the notificationimage generation unit 49 generates graphs for the transitions of theposition coordinates for both the player and the professional golferbased on the position coordinates thus read and generates data of thegraphs and the message of “advice information relating to graph” as thenotification image shown data in FIG. 9.

It should be noted that the notification image data generated by thenotification image generation unit 49 is stored in the RAM 13.

The notification control unit 50 outputs to display on the output unit18 the notification image data generated by the notification imagegeneration unit 49. Here, the notification control unit 50 selects thenotification image data that is generated by the notification imagegeneration unit 49 in response to an input operation by a user via theinput unit 17 and stored in the RAM 13, and outputs to display the dataon the output unit 18.

Notification Processing

Next, notification processing executed by the information notificationapparatus 1 is described.

FIG. 10 is a flowchart showing an example of a flow of notificationprocessing executed by the information notification apparatus 1 of FIG.1 having the functional configuration of FIG. 2.

In FIG. 10, when the input unit 17 of the information notificationapparatus 1 is operated for starting capturing an image of a subject,the processing of Steps S1 to S10 is executed.

In Step S1, the image capture control unit 41 causes the image captureunit 16 to continuously capture a subject (a player) and output theimage data of the subject as moving image data.

In Step S2, the image acquisition unit 42 acquires moving image dataoutput from the image capture unit 16. Then, the image acquisition unit42 encodes the moving image data and stores it in the first imagestorage unit 61.

In Step S3, the image reading unit 43 reads moving image data of theplayer stored in the first image storage unit 61 and moving image dataof the professional golfer stored in advance in the second image storageunit 62. In other words, the image reading unit 43 sequentially decodesthe moving image data stored in the first image storage unit 61 and thesecond image storage unit 62 with each captured image as a unit andstores a sequence of data of the captured images in the RAM 13.

In Step S4, the first detection unit 44 detects, for each of the playerand the professional golfer, a specific swing state image based on themoving image data of the player and the moving image data of theprofessional golfer that is read in Step S3. Here, the specific swingstate refers to the seven states of the “address” state, “halfway back”state, “halfway down” state, “top” state, “impact” state, “follow”state, and “finish” state, as shown in FIG. 4 described later.Furthermore, the first detection unit 44 stores the seven specific swingstate images thus detected in the RAM 13 for each of the player and theprofessional golfer.

In Step S5, the face detection unit 45 detects face images of each ofthe player and the professional golfer for each of the seven specificswing state images detected in Step S4.

In Step S6, the second detection unit 46 detects position coordinates inthe captured images of the face images detected in Step S5 for each ofthe seven specific swing state images. Furthermore, the second detectionunit 46 detects, for each of the player and the professional golfer, thetilt of the face in the right-left direction, the tilt of the face inthe up-down direction, and the tilt of the face in the rotationaldirection, based on the face images detected in Step S5. Furthermore,the second detection unit 46 associates the position coordinates of theface images, the tilt of the face in the right-left direction, the tiltof the face in the up-down direction, and the tilt of the face in therotational direction detected with the seven specific swing state imagesand stores the data in the RAM 13.

Furthermore, in Step S6, the second detection unit 46 detects the sizeof the face image and a position coordinate of a golf ball image in thecaptured image. It should be noted that the second detection unit 46stores the size of the face image and the position coordinate of thegolf ball image in the captured image in the RAM 13 for each of theplayer and the professional golfer.

In Step S7, the comparison unit 47 compares, between the player and theprofessional golfer, the tilts of the face in the right-left direction,the tilts of the face in the up-down direction, and the tilts of theface in the rotational direction, detected by the second detection unit46, and calculates the differences.

In Step S8, the advice information generation unit 48 generates adviceinformation based on each of the difference values (the abovementioned“position advice information” and the like), which is the comparisonresult in Step S7. Furthermore, the advice information generation unit48 associates the “position advice information”, the “advice informationrelating to position and angle difference” and the “advice informationrelating to difference in balls” thus generated with the player's swingstate and stores the data in the RAM 13. Furthermore, the adviceinformation generation unit 48 stores the “advice information relatingto graph” thus generated in the RAM 13.

In Step S9, the notification image generation unit 49 generates thenotification image data shown in FIGS. 5 to 9 based on the adviceinformation generated in Step S8, the seven specific swing state imagesfor each of the player and the professional golfer detected in Step S4,and the position coordinates of the face images, the tilts of the facesin the right-left direction, the tilts of the faces in the up-downdirection, the tilts of the faces in the rotational direction, and theposition coordinates of the golf ball images, which are detected in StepS6. Furthermore, the notification image generation unit 49 stores thenotification image data thus generated in the RAM 13.

In Step S10, the notification control unit 50 outputs to display on theoutput unit 18 the notification image data generated by the notificationimage generation unit 49. Here, the notification control unit 50 selectsthe notification image data that is generated by the notification imagegeneration unit 49 in response to an input operation by a user via theinput unit 17 and stored in the RAM 13, and outputs to display the dataon the output unit 18.

Embodiments of the present invention are described above.

The information notification apparatus 1 executing the abovementionednotification processing includes the image acquisition unit 42, thefirst detection unit 44, the face detection unit 45, the comparison unit47, and the notification control unit 50.

The image acquisition unit 42 acquires a plurality of moving images inwhich the motions of the subjects are captured, respectively.

The first detection unit 44 detects specific swing states of the motionof the subjects from respective moving images among the plurality ofacquired moving images.

The face detection unit 45 detects images corresponding to therespective detected specific swing states from the moving images.

The comparison unit 47 compares the detected images.

The notification control unit 50 notifies information based on acomparison result by the comparison unit 47.

Therefore, information based on a comparison result between an image ofthe player and an image of the professional golfer in a specific swingstate after the swing motion of the player, for example, information ofa comparison result of a position of a face image, is notified.Consequently, it is possible to easily recognize the difference betweenthe player and the professional golfer as subjects in the specific swingstates even not during the golf swing.

The first detection unit 44 detects a plurality of types of specificstates from respective moving images among the plurality of movingimages.

The face detection unit 45 detects a plurality of images correspondingto the plurality of types of respective detected specific states.

The comparison unit 47 compares a plurality of detected images.

Therefore, it is possible to easily recognize the difference between theplayer and the professional golfer as subjects in the plurality ofspecific swing states even not during the golf swing.

The face detection unit 45 detects partial images of subjects fromimages corresponding to the respective detected specific states.

The comparison unit 47 compares the detected partial images.

Therefore, information based on a comparison result between an image ofthe player and an image of the professional golfer in a specific swingstate after the swing motion of the player, for example, information ofa comparison result of a position of a face image, is notified.Consequently, it is possible to easily recognize the difference betweenthe player and the professional golfer as subjects in the specific swingstates even not during the golf swing.

Furthermore, the information notification apparatus 1 includes thesecond detection unit 46.

The second detection unit 46 detects positions or directions of thedetected partial images.

The comparison unit 47 compares the positions or the directions.

Therefore, it is possible to easily recognize the difference of theposition or the tilt of the face between the player and the professionalgolfer as subjects in the specific swing states even not during the golfswing.

Furthermore, the information notification apparatus 1 includes thenotification image generation unit 49.

The notification image generation unit 49 generates notification imagecorresponding to the comparison result of the comparison unit.

Furthermore, the notification control unit 50 notifies notificationinformation of the comparison result with respect to the positions orthe directions, and/or a notification image indicative of the positionsor the directions.

Therefore, it is possible to easily recognize the difference in theposition or the tilt of the face between the player and the professionalgolfer as subjects in a plurality of the specific swing states even notduring the golf swing, and furthermore, it is possible to recognizespecifically the difference in the position nor the tilt of the facebased on the advice information, and it is also possible to visuallyrecognize the position or the tilt of the face based on the mark images.

It should be noted that the present invention is not to be limited tothe aforementioned embodiment, and that modifications, improvements,etc. within a scope that can achieve the object of the present inventionare also included in the present invention.

In the abovementioned embodiment, although there is one informationnotification apparatus 1, the present invention is not limited thereto,and two information notification apparatuses 1 may be provided so as tophotograph a subject from multiple-view points. With such aconfiguration, as shown in FIG. 11, a notification image from which thedifference in the front-back direction can be recognized in a comparisonresult between two specific swing states may be notified.

Furthermore, in the abovementioned embodiment, although the positions ofthe face images of the player and the professional golfer are detected,the present invention is not limited thereto. For example, a position ofan image other than the face, such as an arm and a leg, may be detected.

Furthermore, in the abovementioned embodiment, although the seven statessuch as the “address” state for specific swing states are specified, thepresent invention is not limited thereto, and other states may bedetected. Furthermore, the number of specific swing states may be equalto or less than six or at least eight.

Furthermore, although the cases of applying the present invention aredescribed with the data of a moving image in which the swing of a golfclub is photographed as a target in the abovementioned embodiments, forexample, the present invention can be applied for comparison of variousactions. For example, the present invention can be applied forcomparison of motions of various sports such as the swing of a baseballbat, the swing of a tennis racket, choreography, and the like.

Furthermore, the present invention can be applied for comparison ofvarious motions such as music performance, cooking (knife cuts, handlinga pan, etc.), magic tricks, pottery making, events for the Skill Olympicand Abilympic, etc.

In the aforementioned embodiments, a digital camera has been describedas an example of the information notification apparatus 1 to which thepresent invention is applied; however, the present invention is notparticularly limited thereto.

For example, the present invention can be applied to any electronicdevice in general having an image processing function. Morespecifically, for example, the present invention can be applied to alap-top personal computer, a printer, a television, a video camera, aportable navigation device, a cell phone device, a portable gamingdevice, and the like.

The processing sequence described above can be executed by hardware, andcan also be executed by software.

In other words, the hardware configuration shown in FIG. 2 is merely anillustrative example, and the present invention is not particularlylimited thereto. More specifically, the types of functional blocksemployed to realize the above-described functions are not particularlylimited to the example shown in FIG. 2, so long as the informationnotification apparatus 1 can be provided with the functions enabling theaforementioned processing sequence to be executed in its entirety.

A single functional block may be configured by a single piece ofhardware, a single installation of software, or any combination thereof.

In a case in which the processing sequence is executed by software, aprogram configuring the software is installed from a network or astorage medium into a computer or the like.

The computer may be a computer embedded in dedicated hardware.Alternatively, the computer may be a computer capable of executingvarious functions by installing various programs, e.g., ageneral-purpose personal computer.

The storage medium containing such a program can not only be constitutedby the removable medium 31 shown in FIG. 1 distributed separately fromthe device main body for supplying the program to a user, but also canbe constituted by a storage medium or the like supplied to the user in astate incorporated in the device main body in advance. The removablemedium 31 is composed of, for example, a magnetic disk (including afloppy disk), an optical disk, a magnetic optical disk, or the like. Theoptical disk is composed of, for example, a CD-ROM (Compact Disk-ReadOnly Memory), a DVD (Digital Versatile Disk), or the like. The magneticoptical disk is composed of an MD (Mini-Disk) or the like. The storagemedium supplied to the user in a state incorporated in the device mainbody in advance may include, for example, the ROM 12 shown in FIG. 1, ahard disk included in the storage unit 19 shown in FIG. 1 or the like,in which the program is recorded.

It should be noted that, in the present specification, the stepsdescribing the program recorded in the storage medium include not onlythe processing executed in a time series following this order, but alsoprocessing executed in parallel or individually, which is notnecessarily executed in a time series.

Although some embodiments of the present invention have been describedabove, the embodiments are merely exemplification, and do not limit thetechnical scope of the present invention. Other various embodiments canbe employed for the present invention, and various modifications such asomission and replacement are possible without departing from the spritsof the present invention. Such embodiments and modifications areincluded in the scope of the invention and the summary described in thepresent specification, and are included in the invention recited in theclaims as well as the equivalent scope thereof.

What is claimed is:
 1. An information notification apparatus comprising:a processor which is configured to: acquire a first moving imagecomprising a plurality of first images in which motion of a first objectis captured, and a second moving image comprising a plurality of secondimages in which motion of a second object is captured, wherein the firstimages show the first object in a plurality of postures as the firstobject moves through the motion captured in the first moving image, andthe second images show the second object in a plurality of postures asthe second object moves through the motion captured in the second movingimage, wherein the first object is different from the second object;specify a specific posture from among the plurality of postures; detectan image among the first images corresponding to the specified specificposture, and detect an image among the second images corresponding tothe specified specific posture; and notify information corresponding todifference values between the detected image from among the first imagesand the detected image from among the second images; wherein theprocessor is configured to calculate, as the difference values, adifference value of a tilt angle and position coordinates of a humanface of each object captured in each detected image corresponding to thespecified specific posture; wherein the processor is configured tocalculate, as the tilt angle, at least one of tilts of the face in aright-left direction, tilts of the face in an up-down direction, andtilts of the face in a rotational direction; and wherein the processoris configured to notify, as the information, advice information about ahead position and a head direction corresponding to the differencevalues between each detected image corresponding to the specifiedspecific posture.
 2. The information notification apparatus according toclaim 1, wherein the processor is further configured to: store, in astorage, a plurality of types of pieces of information corresponding toa plurality of types of specific postures, as information based on thedifference values, and select the information based on the differencevalues from the plurality of types of pieces of information, and notifythe selected information.
 3. The information notification apparatusaccording to claim 1, wherein the processor is further configured to:specify a plurality of specific postures from among the plurality ofpostures, detect a plurality of images among the first imagescorresponding to the plurality of specified specific postures,respectively, and detect a plurality of images among the second imagescorresponding to the plurality of specified specific postures,respectively; and calculate the difference values between (i) theplurality of detected images from the first images and (ii)corresponding ones of the plurality of detected images from the secondimages, respectively.
 4. The information notification apparatusaccording to claim 1, wherein the processor is further configured to:specify a plurality of specific postures from among the plurality ofpostures, and detect partial images of the first object from the firstimages corresponding to the respective specified specific postures, andpartial images of the second object from the second images correspondingto the respective specified specific postures, and calculate, as thedifference values, difference values between (i) one of the detectedpartial images of the first object and (ii) a corresponding one of thedetected partial images of the second object.
 5. The informationnotification apparatus according to claim 4, wherein the processor isfurther configured to: detect positions and directions of the detectedpartial images, and calculate, as the difference values, differencevalues between the positions and the directions.
 6. The informationnotification apparatus according to claim 5, the processor is furtherconfigured to: calculate the difference values between the detectedimage from the first images and the detected image from the secondimages; generate a notification image corresponding to the calculateddifference values; and notify notification information of the differencevalues with respect to the positions and the directions, and/or thenotification image indicative of the positions and the directions. 7.The information notification apparatus according to claim 3, wherein theprocessor is further configured to generate a first composite imageformed by combining detected images among the first images correspondingto the plurality of specified specific postures, and a second compositeimage formed by combining detected images among the second imagescorresponding to the plurality of specified specific postures.
 8. Theinformation notification apparatus according to claim 7, wherein theprocessor is configured to generate the first composite image and thesecond composite image so that postures other than a predeterminedposture are transparent relative to the predetermined posture.
 9. Amethod executed by an information notification apparatus, the methodcomprising: acquiring a first moving image comprising a plurality offirst images in which motion of a first object is captured, and a secondmoving image comprising a plurality of second images in which motion ofa second object is captured, wherein the first images show the firstobject in a plurality of postures as the first object moves through themotion captured in the first moving image, and the second images showthe second object in a plurality of postures as the second object movesthrough the motion captured in the second moving image, wherein thefirst object is different from the second object; specifying a specificposture from among the plurality of postures; detecting an image amongthe first images corresponding to the specified specific posture, anddetecting an image among the second images corresponding to thespecified specific posture; and notifying information corresponding todifference values between the detected image from among the first imagesand the detected image from among the second images; wherein the methodfurther comprises: calculating, as the difference values, a differencevalue of a tilt angle and position coordinates of a human face of eachobject captured in each detected image corresponding to the specifiedspecific posture; calculating, as the tilt angle, at least one of tiltsof the face in a right-left direction, tilts of the face in an up-downdirection, and tilts of the face in a rotational direction; andnotifying, as the information, advice information about a head positionand a head direction corresponding to the difference values between eachdetected image corresponding to the specified specific posture.
 10. Anon-transitory computer-readable storage medium having stored thereon acomputer-readable program that is executable by a computer to cause thecomputer to perform functions comprising: acquiring a first moving imagecomprising a plurality of first images in which motion of a first objectis captured, and a second moving image comprising a plurality of secondimages in which motion of a second object is captured, wherein the firstimages show the first object in a plurality of postures as the firstobject moves through the motion captured in the first moving image, andthe second images show the second object in a plurality of postures asthe second object moves through the motion captured in the second movingimage, wherein the first object is different from the second object;specifying a specific posture from among the plurality of postures;detecting an image among the first images corresponding to the specifiedspecific posture, and detecting an image among the second imagescorresponding to the specified specific posture; and notifyinginformation corresponding to difference values between the detectedimage from among the first images and the detected image from among thesecond images; wherein the functions further comprise: calculating, asthe difference values, a difference value of a tilt angle and positioncoordinates of a human face of each object captured in each detectedimage corresponding to the specified specific posture; calculating, asthe tilt angle, at least one of tilts of the face in a right-leftdirection, tilts of the face in an up-down direction, and tilts of theface in a rotational direction; and notifying, as the information,advice information about a head position and a head directioncorresponding to the difference values between each detected imagecorresponding to the specified specific posture.
 11. The informationnotification apparatus according to claim 1, wherein the processornotifies the information as text displayed on a display.
 12. Theinformation notification apparatus according to claim 1, wherein theprocessor is configured to generate the advice information as an imagecorresponding to the difference values, and to notify the image as theadvice information.
 13. The information notification apparatus accordingto claim 1, wherein the processor is configured to: generate anindicator based on the calculated difference values; and notifynotification information of the difference values as the indicatordisplayed on the detected image, wherein the indicator indicates thecalculated difference values.
 14. The information notification apparatusaccording to claim 1, wherein the processor is configured to: notifynotification information corresponding to the calculated differencevalues, in text form.
 15. The information notification apparatusaccording to claim 1, wherein the first object is a first player and thesecond object is a second player different from the first player. 16.The information notification apparatus according to claim 1, wherein theprocessor is configured to: calculate, as the difference values,difference values between (i) the first object in the detected imagefrom among the first images and (ii) the second object in the detectedimage from among the second images; and notify the difference values intext form.
 17. The information notification apparatus according to claim1, wherein the processor is configured to: control a display to display(i) an image corresponding to the information, (ii) the detected imagefrom among the first images and (iii) the detected image from among thesecond images.
 18. The information notification apparatus according toclaim 1, wherein the processor is configured to: detect an imagecorresponding to a timing of the specified specific posture of the firstobject from among the first images, and detect an image corresponding toa timing of the specified specific posture of the second object fromamong the second images; and notify information corresponding to thedifference values between the detected image from among the first imagesand the detected image from among the second images.
 19. The informationnotification apparatus according to claim 1, wherein the processor isconfigured to calculate, as the tilt angle, tilts of the face in theright-left direction, tilts of the face in the up-down direction, andtilts of the face in the rotational direction.